Test and measurement is an important part of modem product development and manufacture. The instruments that perform these tests can typically be automated to perform a number of selected tests. These tests are executed under various conditions on numerous systems or components. The number of items performed by any test sequence depends upon whether the product is still under development or in production, as well as the size of the market for those items in production.
A few basic test techniques are used when controlling a test by an instrument from a computer. One technique is to employ a test program. This technique results in a hand coded program which is inflexible and which is time consuming to change as products change and testing strategies evolve. This technique also requires a level of programming skill which the typical hardware developer does not usually have. This can further increase the time to effect a program change, add to the complexity of creating and verifying test changes, and increase the cost of such a change.
Another technique is to use an interactive soft front panel that provides the user a virtual front panel of the instrument from a computer. However, if this technique is used, typically none of the testing steps are remembered by the computer. No test automation occurs, the user merely uses the computer for control of the instrument. While useful as an exploratory tool, it is ineffective when the test must be repeated as product design changes are made or multiple prototypes are built or in a production or large scale test environment.
Still another technique employs a so-called test executive. A test executive is a program that can control the execution of tests defined in various software components. These software components are in effect small test programs themselves which are activated as needed by the test executive. As products and test strategies evolve, new test components can be added to those that the test executive can execute, and existing test components can be modified as needed. While test executives are more flexible than test programs, they can be more complicated and more involved than the typical user would like.
There are a number of test executives which have been marketed by various companies. Agilent Technologies TestExecSL and FTM-300, National Instrument's TestStand, and Fluke's Spectrum are a few examples. These and other test executive products employ a single sequencing engine to perform the sequencing of tests defined by the user. A consistent problem which these test executives suffer from is that the capabilities of the sequencer are inadequate for some users and much too complex for others. As a result, users have to either settle for less capability than they want or deal with features and capabilities they don't need and don't understand. Since some users may have strong feelings about what capabilities are needed and what capabilities are not needed, those users will find the test executive for their application to be less than optimal.
As with most systems that operate in dynamic environments and that are capable of evolving to meet the challenges of their ever changing environments, test executives tend to have new and modified capabilities added continually. As these features are added, the test executive tends to become more and more complex over time. Most of these additions are visible to the user when they are added as they are then available for use. However, these new features are usually added universally to the user base which is a problem for many users as they will not have a need for the additional functionality. For many users adding new features can be a point of confusion and frustration. In many cases the new features are items that the user does not need and does not wish to be bothered with. At best, users will need to know enough to ignore the new features they don't need. At worst they may need to know how to keep the new features from interfering with their own programs and test sequences. In addition, the only organization that can add these enhancements is the company that provided the test executive. A system integrator who wants to enhance the capabilities of the basic test executive for their customers to add extensions of their own that might provide better service for their particular customer base cannot easily add such capabilities.